Skip to content

ci: pull Docker Hub images through mirror.gcr.io - #930

Open
solace-aross wants to merge 2 commits into
mainfrom
ci/sol-155847-docker-hub-mirror
Open

solace-aross wants to merge 2 commits into
mainfrom
ci/sol-155847-docker-hub-mirror

Conversation

@solace-aross

Copy link
Copy Markdown
Collaborator

CI is failing with Docker Hub 429s (toomanyrequests) when it pulls postgres, pgsty/silo and alpine. Docker Hub allows 100 anonymous pulls per 6 hours per address, GitHub runners share addresses, and one queue run makes about 70 pulls. This blocks every merge.

This is a stopgap until an authenticated Docker Hub account is set up. It adds no secret.

What changes

  • New local action .github/actions/docker-hub-mirror. It merges https://mirror.gcr.io into /etc/docker/daemon.json (existing settings kept), restarts Docker, and prints docker info so the log shows the mirror. It fails the step if the mirror is not in the daemon's config.
  • One step added after checkout in each job that pulls from Docker Hub: test, compat-librdkafka, compat-franz-go, package, smoke, smoke-oldest-client, and test, compat-librdkafka, compat-franz-go in tier-c.yml.
  • package also sets buildkitd-config-inline on docker/setup-buildx-action. BuildKit pulls base images itself and does not use the daemon's mirror, so the alpine pull needs its own setting.

Image tags and digest pins are unchanged.

Verified

  • actionlint 1.7.12 and zizmor 1.30.1 (the workflow-lint settings) report no findings.
  • Mirror serves (HTTP 200) every Docker Hub image CI pulls: postgres 16, 17, 18, 19beta4; alpine 3.24.2 (tag and pinned digest); pgsty/silo (pinned digest); hello-world; apache/kafka 3.7.2, 3.9.2, 4.3.1; moby/buildkit.
  • On a throwaway branch (deleted), the action ran on ubuntu-latest and ubuntu-24.04-arm. docker info listed the mirror, all of those pulls succeeded, and a two-platform buildx build of the alpine base fetched its blobs from mirror.gcr.io.

Downsides

  • The mirror has no SLA or published terms. Google says it keeps images in sync with Docker Hub but does not guarantee a given image stays cached.
  • A mirror miss or outage still falls through to Docker Hub, so it can reproduce the 429. It lowers the pull count against Docker Hub; it does not remove it.
  • Tag-pinned images (postgres, kafka, buildkit) now trust Google's cache as well as Docker Hub. Digest-pinned images (alpine, silo) are still verified against their digest.

Not verified

  • smoke, smoke-oldest-client and package run only in the merge queue, so they are untested until the first queue run. The ubuntu-24.04-arm restart was exercised only on the throwaway branch.
  • Whether the mirror serves a tag nobody has pulled through it yet, such as the next postgres beta.

PR #927 edits the same jobs and will need a rebase over this. The change here is one uses: line per job plus the buildx input.

🤖 Generated with Claude Code

Docker Hub rate limits anonymous pulls per address, and GitHub runners
share addresses, so CI fails with 429s on postgres, silo and alpine.
A local action adds mirror.gcr.io to the Docker daemon's registry
mirrors, and the package job's BuildKit builder gets the same mirror.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Andrea Ross <168456375+solace-aross@users.noreply.github.com>
reubenjds
reubenjds previously approved these changes Oct 10, 2026

@reubenjds reubenjds left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Two nits inline, neither blocking.

Comment thread .github/actions/docker-hub-mirror/action.yml Outdated
Comment thread .github/actions/docker-hub-mirror/action.yml Outdated
The display line failed the step silently under pipefail when no mirror was
set, so the real check never ran. Make the display line non-fatal and have the
check print an error. Read the mirror list into a variable before grepping, so
grep -q closing the pipe early cannot fail docker info with SIGPIPE.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Andrea Ross <andrea.ross@solace.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants